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ABSTRACT 

This study tested a causal model of the influence of 
tutor behavior on student achievement and interest in the context of 
problem-based learning. Data were gathered from 52A tutorial groups 
involving students in the health sciences curriculum at the 
University of Limburg in the Netherlands during 1992-93. Correlations 
among the 261 tutors 1 social congruence, expertise use and cognitive 
congruence behaviors, and small-group functioning and students' 
self-study time, intrinsic interest in the subject matter, and level 
of achievement were analyzed using structural equations modelling. 
The study found that the tutors' level of expertise use and social 
congruence not only directly affected their level of cognitive 
congruence but also affected other elements of the model. The level 
of cognitive congruence influenced tutorial group functioning, which 
in turn affected student self-study time and intrinsic interest. The 
results suggest that subject-matter expertise, a commitment to the 
students' learning and their lives in a personal, authtentic way, and 
the ability to express oneself in the language used by the students 
are all determinants of learning in problem-based curricula. 
(Contains 1A references.) (MDM) 
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Abstract 

Purpose. To test and further develop a causal model of the influence of tutor be- 
haviors on student achievement and interest in the context of problem-based 
learning. Method. Data were analyzed from 524 tutorial groups involving stu- 
dents participating in the four-year undergraduate Health Sciences curriculum a t 
the University of Limburg in 1992-93. These tutorial groups were guided by 261 
tutors. Overall, 3792 data records were studied, with each student participating 
in an average of 2.3 groups. Correlations among tutor's social congruence, expertise 
use and cognitive congruence behaviors, small-group functioning, students' self- 
study time, intrinsic interest in subject-matter and level of achievement were ana- 
lyzed using structural equations modelling. This statistical technique allows the 
investigator to test causal hypotheses on correlational data by comparing the 
structure of these data with a theoretical model. Results. After minor adapta- 
tions, the hypothesized causal model of the effective tutor fitted the data ex- 
tremely well. The tutor's level of expertise use and social congruence not only 
directly affected his or her level of cognitive congruence but also other elements 
of the model. Level of social congruence influenced group functioning in a direct 
fashion, while expertise use had a slightly negative impact on the students' 
level of self-study time and a slightly positive on level of achievement. As hy- 
pothesized, level of cognitive congruence influenced tutorial group functioning. 
Level of group functioning affected self-study time and intrinsic interest. Finally, 
time spent on self-study influenced level of achievement. Conclusion. The results 
suggest that subject-matter expertise, a commitment to the students' learning and 
their lives in a personal, authentic way, and the ability to express oneself in the 
language used by the students are all determinants o: learning in problem-based 
curricula. The theory of the effective tutor, presented in this article, merges two 
different perspectives prevalent in the literature. One perspective emphasizes 
the personal qualities of the tutor; his or her ability to communicate with stu- 
dents in an informal way, coupled with an empathic attitude that enables them 
to encourage student learning by creating an atmosphere in which open exchange 
of ideas is facilitated. The other stresses the tutor's subject-matter knowledge as 
a determinant for learning. The data presented in this article suggests that what 
is needed, really, is much of both. 

Recently, a number of studies have shed light on the role of the tutor in student 
learning in the context of problem-based curricula. In particular, subject-matter exper- 
tise of the tutor has been a focus of attention. 1/2/3,4 a number of studies have dem- 



9 

ERIC 



onstrated effects of tutor expertise on their students' achievement and effort. Davis 

and colleagues^ at the University of Michigan, for instance, showed that student per- 
formance on a test measuring knowledge of influenza was enhanced when their tutors 

entertained an active research interest in that field. Eagle, Harasym and Mandin^ 
demonstrated that students guided by content-expert tutors produced more than twice 
as many learning issues for self-directed learning and spent almost twice the amount of 

time on self-study. Schmidt and colleagues'* found effects of subject-matter expertise 
on achievement in a health sciences curriculum (although these effects were largely con- 
fined to the first curriculum year). Other studies, however, failed to demonstrate no- 
ticeable effects.5/6 According to Schmidt/ this may be due to the fact that subject- 
matter expertise seems tc play a role predominantly when the learning environment 
does not contain sufficient cues as to what is important to study or when students lack 
prior knowledge. Under such unstructured circumstances, students seem to rely on 
their tutor for guidance and might profit if their tutor happens to be someone knowl- 
edgeable regarding the subject under study. 

Less is, however, known about the actual behaviors of tutors that may produce 
these effects on student achievement. Schmidt and colleagues 4 found that subject- 
matter related tutor behaviors, such as guiding students using one's own knowledge, 
were positively correlated with process-facilitative behaviors, such as asking stimulat- 
ing questions, while both were related to achievement. These findings seem to suggest 
that both categories of behaviors make a difference with regard to student achieve- 
ment. Silver and Wilkerson, 3 on the other hand, showed that expert tutors tended to 
take a more directive role in the tutorials, spoke more often and for longer periods, 
provided more direct answers to students' questions, and suggested more items for 
discussion. Tutor-to-student exchanges predominated, with 'ess student-to-student 
discussion. These findings suggest that expert tutors influence achievement at the ex- 
pense of involving students in their own learning (Silver and Wilkerson, however, did 
not report achievement data). Davis et al., 2 finally, were unable to demonstrate any 
differences in behavior between their expert and less-expert tutors. 

In a careful analysis of behaviors displayed by staff and peer tutors involving ex- 
tensive direct observation and interviewing, Moust 8 distinguished between six sets of 
behaviors that can be found among those guiding tutorial groups: Use of subject-matter 
knowledge, use of authority, achievement orientation, an orientation towards co- 
operation in the tutorial group, "role congruence," and "cognitive congruence." He de- 
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fined role congruence as the willingness of the tutor to be "student among the students," 
that is: to seek an informal relationship with the students and display an attitude of 
personal interest and caring. Cognitive congruence was defined as the ability to ex- 
press oneself in the language of the students, using the concepts they use and explain- 
ing things in ways easily grasped by students. Studying student learning in a problem- 
based law curriculum, he found that staff tutors, as compared with student tutors, 
were rated as making more extensive use of their subject-matter knowledge, more 
authoritarian, less cognitively congruent, less role congruent, and less achievement ori- 
ented. He found no differences in co-operation orientation. 

Based on these findings, Moust 8 has proposed a theory of tutor performance and 
how it 'elates to student achievement. A key concept in his theory is cognitive congru- 
ence. Cognitive congruence is a necessary condition for tutors to be effective. If a tutor 
is not able to frame his or her contributions in a language that is adapted to the level of 
the students' understanding of the subject-matter studied, these contributions will go 
unnoticed. In addition, cognitive congruence assumes sensitivity of the tutor concerning 
the difficulties that students may come across while dealing with a problem or with the 
content relevant to that problem. He or she should know when to intervene and what 
to offer: asking for clarification, suggesting a counterexample, or providing some brief 
explanation. A tutor can only be effective in this respect, according to Moust, if he or 
she has relevant subject-matter knowledge and, in addition, has an authentic interest in 
his or her students' lives and their learning. Without appropriate subject-matter 
knowledge it will be difficult to follow the students' line of reasoning, let alone actively 
contribute to it. And without a genuine and personal interest in the students and their 
learning there would not be a tempting reason to help them carrying out their task, nor 
would there be a particular urge to understand the nature of the difficulties students 
meet with while learning based on problems. Therefore, both subject-matter expertise 
and interpersonal qualities are necessary conditions for cognitive congruence to occur. 

Moust framed his ideas in the context of a theory of problem-based learning pro- 
posed by Schmidt and Gijselaers 9 . These authors assume that the tutor's behavior is 
one of three factors affecting the way in which small-group tutorials function (the other 
two being the students' prior knowledge and the quality of the problems handled). In 
turn, the small group's functioning would influence time spent on self-directed learning 
activities and intrinsic interest in the topic studied. Time spent, finally, would influ- 
ence achievement on appropriate tests. Figure 1 summarizes Moust's position on tutor 
behavior and its effects on students. 
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Figure 1. Theoretical model of tutor behaviors and their relationship with other elements 
of problem-based learning. 3 






The figure can be read as follows: The more socially congruent the tutor is, and the 
more he uses his subject-matter knowledge, the more cognitively congruent he will be. 
Higher levels of cognitive congruence cause the tutorial group to function better, which 
expresses itself in more intrinsic interest in subject-matter displayed by the students, 
extended self-study time and higher achievement. The diagram can be considered a 
causal model within the "models-of-school-learning" tradition, exemplified by authors 
such as Bloom 10 and Carroll 11 . 

The purpose of the present paper is to report on the results of a study in which this 
theory of the effective tutor was tested against data gathered in the University of Lim- 
burg's health sciences curriculum. To that end, data from 524 tutorial groups and their 
tutors were studied. These data were analyzed using a structural equations modelling 



a Based on a post-hoc analysis of Moust's data, I decided to merge role congruence and author- 
ity into "social congruence," because they were highly negatively correlated in Moust's study 
and appeared to refer to the same underlying construct; authority implying an aloof attitude 
and, therefore, being the reverse of role congruence. 



approach. 12 Structural equations modelling is a statistical technique that allows the 
investigator to test causal hypotheses on correlational data by comparing the structure of 
these data with a theoretical model. The data may, or may not "fit" the model. 

Method 

Subjects 

Subjects were 1452 students attending the University of Limburg's Health Sciences 
curriculum during the academic year 1992-1993. Each student participated in on average 
2.3 tutorial groups. From the pool of 618 tutorial groups from which data were available, 
524 were selected. The selection criterion was that, for these groups, at least five student 
ratings of their tutors were available. This was 'considered necessary to obtain a reliable 
judgement of the tutors' behavior. In the final sample, 261 tutors were assessed who ran, 
on average, two groups. A total of 3792 data records was included, averaging 7.24 per 
tutorial group. 

Instruments 

Routinely, students responded to a program evaluation questionnaire at the end of 
each course, about two days before the achievement test was taken. This program 
evaluation questionnaire contains items inquiring about the quality of the various ele- 
ments that comprise a course, such as the problems used, the learning resources, the tutor, 
the lectures, the practicals, and so forth. 13 Students are asked to respond to these items 
on a five-point Likert scale ranging from "strongly disagree" to "strongly agree." For the 
present study, tutor behavior was rated on a three-point scale ranging from "not pres- 
ent," via "somewhat present," to "present." 

Social congruence was measured by a five-item rating scale: To what extent the 
tutor took a personal interest in the students' learning activities and the group's well- 
being? To what extent was he or she open to the students' points of view? Use of sub- 
ject-matter expertise was measured by five items, such as: The tutor was sufficiently 
knowledgeable regarding the course's subject-matter. And: The tutor used his subject- 
matter knowledge to guide the group. Cognitive congruence was measured by three 
items, among them: The tutor displayed an understanding of our problems with the 
subject-matter. And: The tutor succeeded in explaining things in a comprehensible 
way. Tutorial-group functioning was measured by two items inquiring whether students 
considered the group productive and whether they thought the meetings were agree- 
able. Time spent on learning was measured by asking students to give an estimate of the 
number of hours per week spent on self-directed learning activities. Student achieve- 
ment was measured after each six-week course by 100-150 true-false items (in the first 



year) and short-essay questions (in subsequent years). The results were transformed to 
a scale ranging from 1 to 4, 3 being the pass score. Finally, intrinsic interest in subject- 
matter was measured by inquiring how interesting they thought the course's subject- 
matter was. Intraclass coefficients as indicators of the generalizability of the ratings 
varied between .75 and .90 (for use of expertise) under the assumption that five raters 
were involved. This finding indicates that the measures were sufficiently reliable for 
use in further analyses. 

Procedure 

For each course, students and tutors were randomly assigned to the groups. The 
groups met twice a week for two hours to discuss the problems presented and to ex- 
change information gathered through self-directed learning. Ratings and achievement 
data were collected at the end of each six-week course. The resulting data were aggre- 
gated for each tutorial group. This was done to ascertain independent measurement. 

Statistical analysis 

The data were analyzed using a structural equations modelling approach. 12 Struc- 
tural equations modelling allows one to test causal hypotheses among multivariate data. 
These theoretical, causal, hypotheses are expressed as a set of structural equations, akin 
to multiple regression functions. EQS, the program used, 12 provides a number of relevant 

statistics, among them a y} statistic that can be used to test whether the empirical data 

sufficiently fit the theoretical model. In addition, other statistics have been developed for 
the evaluation of a particular model. Since educational theories are not yet sufficiently 
developed to allow for all-or-none decisions regarding the acceptability of a certain 
model, often a number of reasonable alternative models are tested, each less stringent 
than its precursor. 

Results and Discussion 
Table 1 displays the correlation matrix for the variables of interest. 
Although the structural equations modelling program analyzes covariances among 
variables rather than correlations, the correlation matrix is given for readability purposes. 



Table 1. Correlations among variables of interest, 
also given. 



Means and standard deviations are 



Expertise Social Cognitive Group Time Achievement Intrinsic 
use congruence congruence functioning spent interest 



Expertise 
use 

Social 

congruence 

Cognitive 

congruence 

Group 

functioning 

Time spent 



1.00 



.55* 



.77* 



.26* 



.07 



1.00 



.71* 



.37* 



.08 



1.00 



.35** 1.00 



.09 



.15* 



1.00 



Achievement .10* 



.08 



.06 



.06 



.21** 1.00 



Intrinsic 
interest 



.16* 



.15* 



.16* 



.44* 



.11* 



.10* 



.00 



Means 



2.60 



2.74 



2.69 



3.86 



16.62 



3.01 



3.90 



Standard 
deviations 



.32 



.26 



.32 



.57 



5.90 



.41 



.61 



* p < .05. **p < .01. 



The model of problem-based learning, outlined in the introduction section, was 
tested against these data. The resulting %^ was equal to 46.89, based on df = 14, p<.001 . 
These findings indicate that the Moust model of the effective tutor does not adequately 
represent the data. A problem, however, with analyses using %^ for the evaluation of 

model adequacy is, that this statistic is quite sensitive to violations of its distribution, in 
particular in relatively small samples. Therefore, other statistics of fit have been devel- 
oped that are less sensitive to violation of assumptions underlying the j(2 distribution. 
These statistics include the Bentler-Bonnett Normed and Nonnormed Fit Indices and the 
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Comparative Fit Index (CFI). Since the CFI takes into account attributes of the unre- 
stricted model relative to the model under test, it will be reported here. For the model 
tested, CFI = .97. In addition, the root mean square residual (RMSR) was smaller than 
.07. 14 The latter two indices suggest that the model tested represents a reasonable first 
approximation of the structure underlying the data. Figure 2 shows the relevant path 
coefficients. These path coefficients indicate the strength of the causal relationship be- 
tween any two variables. Only statistically significant path coefficients are displayed. 

Figure 2. Causal paths for the M<oust model of the effective tutor (error terms are omit- 
ted for readability). 




In comparison to prototypical findings using this technique, the causal influences of 
social congruence and expertise use on cognitive congruence; of cognitive congruence on 
group functioning; and of group functioning on intrinsic interest are fairly large. The 
influence of tutorial group functioning on self-study time is somewhat more limited and 
so is the influence of time-on-task on achievement. Note that in this model only one-to- 
one relations are allowed. This may be an unnecessary restriction, because there is no 
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compelling theoretical reason why, for instance, social congruence of the tutor could not 
influence the quality of tutorial group functioning directly, in addition to an indirect 
influence via cognitive congruence. Assuming that social congruence not only contributes 
to higher levels of cognitive congruence in the tutor, but also may have a direct positive 
impact on the way the group members interact with each other, a direct path would be 
appropriate. In addition, one could assume that the use of expertise by the tutor would 
not only indirectly, through cognitive congruence and group functioning, but also directly, 
affect the amount of time spent by students, or achievement. Etc. We tested some of 
these alternatives and found that with a number of adaptations of the original model an 
excellent fit of the data could be established. The less restrictive model is displayed as 

Figure 3. For this embellished model of the effective tutor, was equal to 15.36, df = 11, 

and p = .17. In addition, CFI = .99 and RMSR < .07. 



Figure 3. Less restrictive variant of the effective-tutor model. 
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These findings complicate, but do not contradict Moust's (1993) original assump- 
tions. Both social congruence and expertise use appear to be important constructs be- 
cause they do not only affect cognitive congruence —as was hypothesized by Moust— but 
also influence other variables in the model. Social congruence does not only help the tutor 
being more cognitively congruent with his or her students, but also seems to facilitate 
group performance in a more direct way. Observations of small-group sessions have 
indeed documented immediate effects of tutoring style on the nature of student interac- 
tions, the more informal tutoring leading to higher levels of participation. 3 In addition, 
students almost invariably report that they feel more free to contribute if a tutor displays 
an interest in what they do. 8 

Intriguing is the slightly negative influence of expertise on time-on-time on task, sug- 
gesting that the more the tutor contributes to the discussion using his own subject-matter 
knowledge, the less time students spend on self-directed learning. A similar trade-off 
effect has been demonstrated between tutorial group functioning and time-on-task 
(though not in the present study), suggesting that students tend to compensate for more 
extensive forms of education — be it through the quality of their tutorial sessions, be it 
through direct intervention of their tutors— by decreasing the amount of time they spent 
on self-study. 9 Finally, the effect of the tutor's subject-matter expertise on achievement 
has been demonstrated in other studies as well. 2 - 4 

The study presented here has several shortcomings, the most notable being that 
students were used as raters of their tutor's behavior. Although students are in the best 
position to observe a tutor for prolonged periods of time, their role is not to observe their 
tutor but to learn. This may have narrowed their perspective on their tutor's functioning. 
A further difficulty with the use of student ratings is, that students may share an implicit 
theory of effective tutoring that not necessarily matches what really is go»ng on in tutorial 
groups. In the latter case, the findings may reflect this implicit theory of tutoring rather 
than reality. However, the fairly high intraclass coefficients found suggest that whatever 
students may have observed, their agreement on what they have seen is considerable. In 
addition, the relationship of the student ratings with achievement found in this study and 
several others is an argument in favor of accurate observation by student raters and, 
hence, of the validity of the data. 

A second, more serious, limitation of the study is the retrospective nature or the 
measurements. The variables of the model were measured at the end of a six-week 
learning cycle. Generally, it is to be preferred to measure behaviors when they occur 
rather than to ask subjects to report "?trospectively on them. Retrospective observation 
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is generally more sensitive to possible Halo effects than "on-line" observation because of 
the human tendency to restructure the past such that "everything makes sense." 

A final and related limitation of the study is that most variables, with the excep- 
tion of achievement were measured concurrently. This poses a problem for causal models 
that assume —at least theoretically— some sort of temporal order in the occurrences of 
events and, therefore, in their measurement. It is, however, extremely difficult to measure 
variables in the predicted temporal order in an ongoing educational process in which 
some of the influences may be reciprocal, others circular. "True" temporally ordered 
measurement may only be possible in experimental designs. 

These considerations, of course, limit the significance of the findings and, at the 
same time, show avenues for further research into the issue of effective tutoring. It would, 
for instance, be useful to study tutors and their effects on student learning using direct 
observation by independent raters. In addition, it would be important to develop meas- 
ures other than retrospective ratings for some of the variables in the model. This would 
be particularly useful for time-on-task and tutorial group functioning, because of their 
central role, not only in the model of effective tutoring presented here, but also in the more 
& eneral model of problem-based learning as developed by Schmidt and Gijselaers. 9 

Conclusion 

Effective tutoring in the context of problem-based learning seems to imply three dis- 
tinct, though interrelated, qualities; the possession of a suitable knowledge base with 
regard to the topic under study, a willingness to become involved with students in an 
authentic way, and the skill to express oneself in a language understood by students. The 
present study has demonstrated how these skills interact with each other and how they 
influence other elements of learning in a problem-based environment: Tutorial-group 
functioning, self-study time and academic achievement. The findings corroborate and 
extend those from an earlier study, using different measurements. 4 In the latter study, it 
was shown that use of subject-matter expertise and student-facilitation behaviors were 
both causally related to achievement and, in addition, were positively correlated. The 
present study demonstrates xohy this is so. 

This theory of the effective tutor merges two different perspectives prevalent in the 
literature. One perspective emphasizes the personal qualities of the tutor; his or her 
ability to communicate with students in an informal way, coupled with an empathic 
attitude that °nables them to encourage student learning by creating an atmosphere in 
which open exchange of ideas is facilitated. 3 - 8 The other stresses the tutor's subject- 
matter knowledge as a determinant for learning. 2 The data presented in this article 
suggests that what is needed, really, is a lot of both. 
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This theory has implications both for the selection and the training of tutors. Rele- 
vant subject-matter knowledge is an obvious selection criterion of tutors for a particular 
course. Previous research has shown that particularly in unstruc'ured learning environ- 
ments and in situations in which the students lack relevant prior knowledge, content 
expertise of the tutor makes a difference. 7 Training should concentrate on methods by 
which tutors can create an informal learning environment in which students feel free to 
exchange ideas with their peers and their tutor. Particular attention should be given to 
cognitive congruence: How can tutors use their subject-matter expertise in helping student 
to come to grips with the topics studied? 
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